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SUPPLEMENTAL  GUIDE:  SOURCES  OF  INFORMATION  FOR  ON-LINE  IMPLEMENTATION  OF 
ISD  1.5  SELECT  INSTRUCTIONAL  SETTING 


BRIEF 


Requirement: 

The  purpose  was  to  develop  a  language  to  translate  an  existing  paper  and 
pencil  Job  Aid  onto  a  computerized  delivery  system.  The  Job  Aid  is  one  of  a 
series  developed  previously  to  support  users  of  the  Instructional  Systems 
Development  Model  (ISD) . 


Procedure : 

A  Programming  Design  Language  (PDL)  was  created  to  describe  the  computer 
functions  (e.g.,  computer/user  interactions,  storage/retrieval  of  data,  program 
branching,  program  management,  and  calculations)  required  by  the  Job  Aids  (ARI 
Research  Produces  80-13  through  80-18).  The  PDL  was  designed  to  communicate 
to  the  computer  programmer  in  a  language  independent  fashion  so  that  the  on-line 
or  computer  version  of  the  Job  Aid  could  be  delivered  by  any  computer.  This 
document  was  designed  to  support  the  user  of  the  computer-based  Job  Aid. 


Utilization: 

The  Programming  Design  Guide  may  be  used  by  computer  programmers  who  are 
tasked  with  programming  the  manual  Job  Aids.  This  supplemental  guide  is 
intended  for  use  by  Instructional  Developers  who  may  use  the  on-line  Job  Aid. 
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Chapter  I 


INTRODUCTION 


Job  aids  are  being  developed  for  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  under  contract  DAHC19-78-C-0010.  The  job  aids  are  intended  to  be 
stand-alone,  step-by-step  procedural  guides  which  are  equally  useful  to  individuals  at  all 
experience  levels  of  the  instructional  systems  development  process. 

There  are  thirteen  job  aids  presently  available.  They  are: 

•  Job  Aid  for  Selecting  Tasks  for  Training 

•  Job  Aid  for  Conducting  Task  Analysis 

•  Job  Aid  for  Analyzing  Existing  Courses 

•  Job  Aid  for  Selecting  Instructional  Setting 

•  Job  Aid  for  Developing  Objectives 

•  Job  Aid  for  Developing  Tests 

•  Job  Aid  for  Describing  Entry  Behavior 

•  Job  Aid  for  Determining  Sequence  and  Structure 

•  Job  Aid  for  Specifying  Learning  Events  and  Activities 

•  Job  Aid  for  Specifying  Instructional  Management  Plan  and  Delivery  System 

•  Job  Aid  for  Review  and  Selection  of  Existing  Materials 

•  Job  Aid  for  Developing  Instruction 

•  Job  Aid  for  Validating  Instruction 

In  addition  to  printed  versions  of  the  above  job  aids,  the  Job  Aid  for  Selecting 
Instructional  Setting  can  be  made  available  in  a  computer  version.  A  Programming  Design 
Guide1  is  available  for  use  by  computer  programmers  which  provides  all  of  the  guidance 
necessary  for  them  to  implement  the  job  aid  on  their  computer  system.  Instructional 
development  personnel  would  then  use  the  on-line  version  of  the  aid  to  assist  in  the  selec¬ 
tion  of  instructional  sotting  for  each  critical  task.  This  document  provides  supplemental 
guidance  for  the  user  of  the  on-line  job  aid.  it  is  not  intended  to  be  a  stand-alone 
publication, 

THIS  SUPPLEMENTAL  GUIDE  IS  NOT  INTENDED  TO  BF  A 

STAND-ALONE  DOCUMENT.  IT  IS  INTENDED  TO  PROVIDE 

ONLY  SUPPLEMENTAL  INFORMATION  THAT  MAY  BE  USEFUL 

TO  INSTRUCTIONAL  DEVELOPMENT  PERSONNEL  WHO  USE 

THE  ON-LINE  VERSION  OF  THE  JOB  AID  SELECT  INSTRUC¬ 
TIONAL  SETTING. 

1  Schulz,  R.E.,  Underhill,  W.G.,  Hargun,  C.S.  Programming  Design  Guide  for  Computer  Imple¬ 
mentation  of  Job  Ai:i  for  Selecting  Instructional  Settings,  (Research  Product  8Q-24b).  Alexandria,  VA: 
US  Army  Research  Institute,  August  I960. 


Chapter  II 

SOURCES  OF  INFORMATION 


OVERVIEW  OF  THE  INSTRUCTIONAL  SETTINGS  JOB  AID 
•  Objective 

1.  Given  a  list  of  tasks  selected  for  training  within  a  single  skill  level,  select  the  most 
appropriate  instructional  setting  for  training  each  task  to  the  Soldier's  Manual 
Standard.  (Qualification  Training) 

2.  Record  the  basis  for  each  instructional  setting  selection. 


•  Purpose 

The  purpose  of  this  aid  is  to  help  you  choose  instructional  settings  (training  locations) 
for  tasks  selected  for  training  within  each  skill  level.  Due  to  advancements  in  instructional 
technology  it  is  often  more  cost-beneficial  and  efficient  to  train  tasks  in  a  non-institutional 
(extension)  setting.  This  aid  is  designed  to  help  you  identify  as  many  tasks  as  possible 
for  which  extension  training  is  appropriate. 


•  Product 

This  job  aid  will  result  in  a  listing  of  all  critical  tasks  in  which  each  task  is  assigned  for 
training  to  one  of  the  following  instructional  settings: 

a.  Institution  (Resident  school  training) 

b.  Supervised  On-the-Job  Training  (SOJT) 

c.  Self-study 

This  output  will  be  especially  useful  in  the  preparation  of  the  Commander’s  Manual. 


•  Overview  of  Major  Steps  in  Selecting  Instiuctional  Setting 

Step  1.  Tasks  selected  for  training  are  categorized  by  skill  level  and  duty  position. 

Step  2.  Task  performance  data  is  obtained  and  recorded  for  each  task.  That  is: 

a.  In  which  duty  nos’tion  is  the  task  performed? 

b.  What  percentage  of  soldiers  perform  the  task? 
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Step  3.  Initial  assignment  of  the  task  to  one  of  three  instructional  settings  (institu¬ 
tion,  supervised  on-the-job  training,  or  self-study)  is  made  based  on  the 
answers  to  the  following  14  questions: 

(1)  Is  task  a  common  skill  level  task? 

(2)  Is  task  performed  by  a  high  percentage  of  soldiers? 

(3)  Is  tusk  performed  in  a  similar  manner  in  various  duty  positions 
and  units? 

(4)  Is  proficiency  in  task  performance  retained  over  time  (that  is,  not 
easily  forgotten)? 

(5)  Does  task  require  considerable  theoretical  knowledge? 

(6)  Must  the  task  be  performed  immediately  on  entry  to  the  job? 

(7)  Is  the  task  a  prerequisite  for  learning  to  perform  other  school 
trained  tasks? 

(8)  Is  training  equipment  and/or  facilities  only  available  at  the  school? 

(9)  Is  the  equipment  required  for  individual  training  of  the  task  in  the 
unit,  available  at  most  units? 

(10)  Are  personnel  with  the  necessary  exoertise  to  conduct  training  oi 
the  task  available  at  most  units? 

(11)  Do  operational  requirements  at  most  units  allow  sufficient  time  for 
the  soldier  to  receive  training  in  the  unit? 

(12)  Can  the  task  be  learned  with  very  little  supervision? 

(13)  Does  the  soldier’s  schedule  allow  sufficient  tune  for  independent 
study? 

(14)  Can  everything  required  for  training  (which  is  not  already  available 
in  the  unit)  be  included  in  the  exportable  training  package  at  a  cost 
competitive  with  school  I  institution)  training? 

Step  4.  Administrative  review  and  final  selection  >f  instructional  setting. 

In  Step  3,  tasks  are  assigned  initially  to  one  of  the  three  instructional  settings. 
In  this  step  each  task  is  reviewed  to  determine  if  the  initial  assignment  is 
still  the  best  instructional  setting  on  the  basis  of  expert  opinion.  I.i  the  review 
of  each  task,  questions  of  the  type  shown  on  the  following  displays  are  asked: 

(1)  Have  so  many  tasks  been  assigned  ,  SOJT  or  Self-Study  that  -here  is 
not  enough  time  to  train  all  the  tasks  before  they  must  be  performed? 


(2)  Have  so  many  tasks  been  assigned  to  SOJT  that  units  can’t  handle 
the  load? 

(3)  Have  so  few  tasks  been  assigned  to  a  particular  setting  that  adminis 
trative  costs  outweigh  the  advantages  of  training  so  few  tasks  in  this 
instructional  setting? 

(4)  Would  resource  and  time  constraints  in  the  development  of  new  train¬ 
ing  programs  delay  the  onset  of  critically  needed  training? 

(5)  Are  there  any  other  reasons  why  the  initially  selected  instructional 
setting  should  be  changed? 

After  reviewing  each  task  and  asking  questions  similar  to  those  just  shown, 
any  indicated  changes  in  instructional  setting  is  made.  The  rationale  for 
each  change  is  to  be  carefully  documented. 

The  task  listing  with  the  final  selections  of  instruction  settings  is  submitted 
to  the  supervisor  for  review  and  revision. 


DESCRIPTION  OF  INSTRUCTIONAL  SETTINGS 


Supervised  On-Thedob 

Institution  (INST) 

Training  (SOJT) 

Self-Study 

Training  conducted  at  TRADOC 

Training  conducted  at  the 

Training  administered  during 

resident  schools  and  includes: 

OSUT  •  One  Station  Unit 
Training 

PNCOC  -  Primary  Noncom¬ 
missioned  Officer 
Course 

BNCOC  -  Basic  Noncom¬ 
missioned  Officer 
Course 

soldier's  unit 

individuals  own  time,  usually 
at  the  soldier's  unit  and  includes: 
Self-teaching  exportable 
packages  (STEP) 

Training  Extension  Courses 

Job  Performance  Aids 

Study  Guides 

Correspondence  Courses 

Films,  tapes,  etc. 

ANCOC  •  Advanced  Noncom 

missioned  Officer 
Course 

Training  always  conducted  under 

Training  supervised  by  best 

Little  supervision  required 

supervision  of  qualified 
instructors 

qualified  NCOs  in  unit 
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ADVANTAGES  AND  DISADVANTAGES  OF 
THREE  INSTRUCTIONAL  SETTINGS 


Instructional  Setting 

Advantages 

Disadvantages 

INSTITUTION 

Usually  best  setting  for  training 
common  skill  level  tasks  or 
tasks  that  are  performed  by 
large  percentage  of  soldiers  in 
the  MOS/skill  level 

Sophisticated  training  resource 
and  eroertise  available 

Lack  of  real  world  environment 

High  cost  of  soldier's  housing 
and  travel 

Time  is  spent  away  from  job 
assignment 

SUPERVISED 

ON-THE-JCB 

TRAINING 

Effective  for  training  ta'^s  that 
can  be  learned  faster  or  better 
with  hands-on  experience 

Soldier  contributes  to  unit's 
mission  while  learning 

May  tie  up  unit’s  equipment  and 
thus  equipment  may  not  be  avail¬ 
able  for  operational  use 

May  overburden  supervisors 

Reduces  time  available  in  unit  for 
operational  requirements 

SELF-STUDY 

Effective  for  training  tasks 
which  can  be  learned  without 
an  instructor  or  where  little 
supervision  is  required 

If  study  occurs  during  normal 
duty  hours,  this  type  of  training 
may  reduce  time  available  in  unit 
for  operational  requirements 

Can  be  accomplished  at 
trainee's  convenience 

May  require  soldier  to  devote 
considerable  off-duty  time  to 
study 
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SOURCES  OF  INFORMATION  FOR  DETERMINING  PERCENTAGE  OF 
SOLDIERS  IN  THE  SKILL  LEVEL  WHO  PERFORM  EACH  TASK 


CODAP  Group  Summary 

CODAP  data,  when  available,  are  excellent  for  determining  the  percent  of  soldiers 
within  the  skill  level  who  perform  each  task.  However,  it  is  likely  that  not  all  tasks 
represented  in  the  skill  level  will  be  represented  on  the  CODAP  Report.  Consequently, 
even  when  a  CODAP  Report  is  available,  it  will  probably  be  necessary  to  supplement  the 
data  from  the  Report  with  data  from  other  sources. 


Field  Survey 

Field  Survey  data  are  excellent  for  determining  the  percent  of  soldiers  who  perform 
each  task.  However,  a  field  survey  should  only  be  conducted  when  CODAP  data  are  not 
available  (or  badly  out  of  date)  and  when  there  is  sufficient  time  to  conduct  the  survey. 
Guidance  for  conducting  a  field  survey  can  be  found  in  ~’iapter  III  of  this  guide. 


Panel  of  Recent  Job  Incumbents 

This  represents  a  fair  source  of  information  for  determining  the  percent  of  soldiers 
within  the  skill  level  who  perform  each  task.  See  Chapter  IV  for  guidance  in  establishing 
and  using  a  panel  of  recent  job  incumbents. 


Panel  of  Subject  Matter  Experts 

Use  this  source  only  if  none  of  the  above  sources  are  available.  See  Chapter  IV 
for  guidance  in  establishing  and  using  this  type  of  panel. 

Your  Own  Judgment 

As  a  last  resort,  you  may  have  to  use  your  own  judgment. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  TRAINING 
REQUIREMENTS  FOR  THE  TASK  ARE  ESSENTIALLY  THE  SAME 
REGARDLESS  OF  THE  MISSION,  EQUIPMENT  ALLOCATION, 
GEOGRAPHICAL  LOCATION,  ETC.,  OF  UNITS  IN  WHICH  THE 
JOB  INCUMBENT  IS  ASSIGNED 


—  Job  performance  measures  or  task  performance  descriptions  developed  in  ISD  1.3 
Construct  Job  Performance  Measures  is  an  excellent  source  o!  equipment  used  in 
task  performance.  This  will  assist  in  determining  whether  equipment  differences 
between  units  will  have  an  effect  on  training  requirements. 
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—  TOE/MTOE  and  TDA  is  another  excellent  source  of  information  concerning 
equipment  allocation  in  various  units. 

—  Training  Manuals  and  supply  bulletins  used  in  conjunction  with  TOE  should  be 
considered  as  a  good  sc  urce  of  information. 

—  Panel  of  recent  job  incumbents  can  provide  good  information  for  answering 
question  3.  See  Chapter  TV  for  guidance  in  establishing  and  using  this  particular 
panel. 

—  Panel  of  subject  matter  experts.  This  also  is  a  good  source.  See  Chapter  IV  for 
guidance  in  establishing  and  using  a  panel  of  subject  matter  experts. 

—  Your  own  judgment.  Use  only  as  a  last  iesort  or  in  conjunction  with  other  sources. 


EXAMPLES  OF  FACTORS  TO  CONSIDER  IN  DETERMINING  IF  A 
TASK  TAUGHT  IN  THE  SCHOOL  (INSTITUTION)  WILL  STILL  BE 
REMEMBERED  BY  THE  TIME  THE  SOLDIER  HAS  TO  PERFORM 
THE  TASK  ON  THE  JOB 

—  Tasks  for  which  the  soldier  has  had  previous  civilian  or  military  experience  will 
usually  be  easily  remembered  (that  is,  driving  a  vehicle). 

—  Tasks  which  the  soldier  considers  important  to  remember  will  be  belter  learned 
and  more  easily  recalled. 


—  Some  tasks  require  frequent  opportunities  for  practice  in  order  to  retain  task 
proficiency. 

—  In  general,  motor  tasks  (physical  activities)  are  more  easily  remembered  than 
mental  tasks. 


—  Tasks  which  involve  short  regular  procedures  arc  more  easily  remembered  than 
those  for  which  there  is  no  clear  cut  procedure  to  follow. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  THERE  IS  A 
CONSIDERABLE  AMOUNT  OF  THEORY  TO  BE  TAUGHT  WITH  THE 
TASK  UNDER  CONSIDERATION 
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—  Output  from  1SD  1.3  Construct  Job  Performance  Measures  or  1SD  1.2  Conducting 
Task  Analysis  provide  a  good  source  of  information.  Examine  task  descriptions 
to  determine  how  the  task  is  performed.  This  should  provide  an  excellent  insight 
into  the  amount  of  theoretical  content  that  will  be  required  for  training  the  task. 

—  Training  Manuals  are  an  excellent  source  of  information. 
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—  Field  survey  of  Job  Supervisors  is  an  excellent  source  if  time  for  survey  is  avail¬ 
able.  See  Chapter  III  for  guidance  in  conducting  a  field  survey. 

—  Panel  of  Recent  Job  Supervisors.  Fair  Source.  See  Chapter  IV  for  guidance 
in  establishing  panel. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  THE  TASK 
MUST  BE  PERFORMED  IMMEDIATELY  ON  ENTRY  TO  THE  JOB 

—  Field  Survey  of  Job  Supervisors  and/or  Incumbents.  Excellent  source  when  time 
for  survey  is  available  and  if  information  is  not  available  through  CODAP.  See 
Chapter  ID  "or  guidance  in  conducting  the  survey. 

—  Panel  of  Recent  Job  Incumbents.  Good  Source.  See  Chapter  IV  for  guidance  in 
selecting  panel. 

—  Your  ovm  judgment.  Use  only  if  all  other  sources  are  unavailable. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  THE  TASK  IS  A 
PREREQUISITE  FOR  LEARNING  AND/OR  PERFORMING  OTHER  SCHOOL 
TRAINED  TASKS 

—  Check  Training  Manuals,  task  description  etc.,  to  determine  training  equipment 
requirements.  Survey  field  supervisors  to  determine  if  training  equipment  is 
available  in  field  units. 

—  Panel  of  Recent  Job  Supervisors.  Use  in  conjunction  with  Training  Manuals, 
Job  Performance  Measures,  etc.  See  Chapter  IV  for  guidance  in  selecting  panel. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  EQUIPMENT 
REQUIRED  FOR  INDIVIDUAL  TRAINING  OF  THE  TASK  IN  THE  UNIT 
IS  AVAILABLE  AT  MOST  UNITS 

—  Field  Survey  of  Supervisors  is  an  excellent  source  of  information  if  time  for  sur¬ 
vey  is  available.  See  Chapter  IV  for  guidance  in  conducting  a  survey. 

—  Panel  of  Recent  Job  Supervisors  is  a  good  source  of  information.  See  Chapter  IV 
for  guidance  in  selecting  panel. 

—  Training  Manuals  to  determine  equipment  requirement  followed  by  review  of 
TOE/MTOE  or  TDA  for  appropriate  units.  Good  source  of  information. 
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SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  PERSONNEL  WITH 
THE  NECESSARY  EXPERTISE  ARE  AVAILABLE  AT  MOST  UNITS  TO 
CONDUCT  THE  TRAINING  FOR  THE  TASK 


—  Field  Survey  of  Supervisors  is  an  excellent  source  of  information  if  time  for  survey 
is  available  See  Chapter  III  for  guidance  in  conducting  a  field  survey. 

—  Panel  of  Recent  Job  Supervisors.  Good  Source.  See  Chapter  IV  for  guidance 
in  convening  the  panel. 

—  Your  own  judgment.  Use  only  if  other  sources  are  not  available. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  OPERATIONAL 
REQUIREMENTS  AT  MOST  UNITS  ALLOW  SUFFICIENT  TIME  FOR 
THE  SOLDIER  TO  BE  TRAINED  IN  THE  UNIT 

—  Field  Survey  of  Supervisors  is  an  excellent  source  of  this  information.  See 
Chapter  III  for  guidance  in  conducting  a  field  survey. 

—  Panel  of  Recent  Job  Supervisors.  Good  source.  See  Chapter  IV  for  guidance 
in  selecting  panel. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  TASK  CAN  BE 
LEARNED  WITH  VERY  LITTLE  SUPERVISION 

—  Output  from  ISD  1.2  Select  Task/Functions.  Task  learning  difficulty  should  have 
been  established  on  a  rating  scale  of  I  to  7  and  will  therefore  be  an  excellent 
indication  of  the  amount  of  supervision  required. 

—  Field  Survey  of  Job  Supervisors.  Excellent  source  if  time  for  survey  is  available. 
See  Chapter  HI  for  guidance  in  conducting  field  survey. 

—  Panel  of  Recent  Job  Supervisors.  Good  source  of  information.  See  Chapter  IV 
for  guidance  in  selecting  panel. 

—  Your  own  judgment.  Use  only  if  other  sources  are  not  available. 


SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  THE  SOLDIER'S 
SCHEDULE  ALLOWS  SUFFICIENT  TIME  FOR  INDEPENDENT  STUDY 


—  Field  Survey  of  Job  Incumbents  and  Job  Supervisors.  Excellent  source  especially 
when  a  comparison  is  made  between  the  two  sources.  See  Chapter  III  for 
guidance  in  conducting  a  field  survey. 
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—  Panel  of  Recent  Job  Incumbents  and  Recent  Job  Supervisors.  Good  source.  See 
Chapter  IV  for  guidance  >n  selecting  panels. 

—  Your  own  judgment.  Use  only  if  other  sources  are  not  available. 
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SOURCES  OF  INFORMATION  FOR  DETERMINING  IF  EVERYTHING 
REQUIRED  FOR  TRAINING  (WHICH  IS  NOT  ALREADY  AVAILABLE 
IN  THE  FIELD)  CAN  BE  INCLUDED  IN  A  TRAINING  PACKAGE  AND  IS  IT 
INEXPENSIVELY  EXPORTABLE 


—  Check  with  Course  Development  Personnel 

—  Panel  of  Subject  Matter  Experts.  Fair  source.  See  Chapter  IV  for  guidance  in 
selecting  panei. 

—  Your  own  judgment.  Use  only  if  other  sources  are  not  available. 


FACTORS  TO  CONSIDER  WHEN  RE-EXAMINING  THE  TASKS 
INITIALLY  ASSIGNED  TO  AN  INSTITUTIONAL  INSTRUCTIONAL 
SETTING 

•  For  each  task  initially  assigned  to  the  Institution  consider  any  reason  why  the 
task  should  NOT  be  trained  in  an  institutional  setting.  Following  are  examples 
of  questions  you  might  want  to  ask  as  you  review  each  task: 

—  Do  feedback  from  the  field  or  SQT  results  indicate  that  an  Institutional  setting 
has  proven  ineffective  for  any  of  these  tasks? 

—  Are  appropriate  cues  or  stimuli  not  available  in  the  school  for  any  task? 

—  Are  skilled  instructors  available  for  teaching  the  task  in  the  school? 

—  Is  time  and  money  available  for  training  the  task  in  the  school? 

—  Are  there  any  job  factors  unique  to  this  MOS  which  would  cause  you  to  change 
this  instructional  setting? 

—  Will  new  equipment/simulators  soon  to  be  available  cause  you  to  change  this 
setting? 

You  may  have  other  reasons  why  the  task  should  not  be  taught  at  the  insti¬ 
tution.  Consider  each  reason  carefully. 


FACTORS  TO  CONSIDER  WHEN  RE-EXAMINING  THE  TASKS  INITIALLY 
ASSIGNED  TO  AN  SOJT  INSTRUCTIONAL  SETTING 

•  For  each  task  initially  assigned  to  SOJT  consider  any  reason  why  the  task  should 
NOT  be  trained  in  a  SOJT  setting.  Following  are  examples  of  questions  you 
might  want  to  ask  as  you  review  each  task. 

—  Do  feedback  from  the  field  or  SQT  results  indicate  that  a  SOJT  setting  has 
proven  ineffective  for  any  of  these  tasks? 
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—  Is  the  environment  too  hazardous  for  training  this  task  in  the  field?  For 
example,  would  errors  made  during  training  in  a  field  environment  be  critical 
to  the  student,  to  others,  or  to  equipment? 

-Will  new  equipment/simulators  soon  be  available  which  would  cause  you  to 
change  u,e  setting? 

—  Do  “percentage  performing”  figures  indicate  that  fewer  soldiers  are  performing 
this  task  than  should  be?  If  so,  does  the  percent  which  should  be  performing 
meet  the  criterion  for  high  task  performance  and  therefore  consideration  for 
Institutional  training? 

—  Are  them  any  job  factors  unique  to  this  MOS  which  would  cause  you  to  change 
this  instructional  setting? 

You  may  have  other  reasons  why  the  task  should  not  be  taught  by  SOJT. 
Consider  ea  h  reason  carefully. 


FACTORS  TO  CONSIDER  WHEN  RE-EXAMINING  THE  TASKS  INITIALLY 
ASSIGNED  TO  A  SELF  STUDY  INSTRUCTIONAL  SETTING 

•  For  each  task  i  uually  assigned  to  Self-Study  consider  any  reason  why  the  task 
should  NOT  he  trained  in  a  Self-Study  setting.  Following  are  examples  of  ques¬ 
tions  you  might  want  to  ask  as  you  review  each  task:  ~  ~ 

—  Do  feedback  from  the  field  or  SQT  results  indicate  that  a  Self-Study  setting 
has  proven  ineffective  for  any  of  these  tasks? 

—  Will  new  equipnn mt/simulators  soon  be  available  which  would  cause  you  to 
change  the  setting? 

—  Do  “percentag"  performing”  figures  indicate  that  fewer  soldiers  are  performing 
this  task  than  should  be?  If  so,  does  the  percent  which  should  be  performing 
meet  the  criteric;?  for  high  task  performance  and  therefore  consideration  for 
Institutional  training? 

—  Would  the  percent  performing  figure  change  radically  during  mobilization? 
Would  the  task  then  become  a  candidate  for  training  in  the  Institution? 

—  Are  there  any  job  factors  unique  to  this  MOS  which  would  cause  you  to  change 
this  instructional  setting? 

You  may  have  other  reasons  why  the  task  should  not  be  taught  by  Self-Study. 
Consider  each  reason  carefully. 
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Chapter  III 

CONDUCTING  FIELD  SURVEYS 


WHAT  IS  A  FORMAL  FIELD  SURVEY? 

A  formal  field  survey  is  similar  to  the  type  of  questionnaire  survey  conducted  by 
the  Army  Occupational  Survey  Program,  only  it  is  conducted  by  instructional  develop¬ 
ment  personnel  within  an  Army  service  school. 

The  use  of  questionnaires  permits  the  job  analysis  team  to  make  limited  contact 
with  large  numbers  of  personnel;  thus  large  amounts  of  information  can  be  collected  at 
a  relatively  low  cost.  Questionnaires  can  be  mailed  to  personnel  who  are  asked  to  com¬ 
plete  and  return  them,  or  they  can  be  administered  to  groups  of  job  incumbent"'  and/or 
supervisors  by  local  personnel  who  have  the  responsibility  and  authority  to  make  sure  all 
questionnaires  are  completed  and  returned. 


WHEN  SHOULD  FORMAL  FIELD  SURVEYS  BE  USED? 

Whenever  time  does  not  allow  you  to  access  information  from  the  Army  Occupa¬ 
tional  Survey  Program,  an  alternate  data  collection  method  may  be  used.  Formal  field 
surveys  are  suggested  as  an  alternate  data  source  in  the  ISD  Job  Aids.  Should  you  decide 
to  conduct  a  formal  field  survey  be  sure  to  obtain  permission  from  MILPERG'EN  in 
accordance  with  the  guidelines  in  AR  600-46. 


HOW  IS  A  FIELD  SURVEY  QUESTIONNAIRE  DESIGNED? 

•  Types  of  Questionnaires 

There  are  two  types  of  questionnaires,  the  closed  form  and  the  open  form.  We 
suggest  using  the  closed  form,  which  contains  a  list  of  possible  items  to  be  selected  or 
blanks  to  be  filled  in  with  words  or  numbers.  (In  the  open  form  questionnaire  the 
respondent  might  be  asked  “list  the  tasks  you  now  perform  and  rate  the  amount  of  time 
spent  on  each  task.”  For  an  example  of  a  closed  form  questionnaire,  see  pages  14  and  15. 
This  form  has  several  advantages  over  the  other  alternative,  the  open  form.  It  is  likely 
to  take  a  minimum  amount  of  time  to  fill  out,  thus  increasing  the  chances  that  it  will 
be  completed  and  returned.  The  process  of  tabulating  and  summarizing  responses  is 
simpler  and  less  time  consuming  than  with  an  open  form  questionnaire.  Machine  tabu¬ 
lation  and  computer  analysis  of  the  completed  forms  arc  practical  when  a  large  number 
of  questionnaires  is  used. 

A  properly  designed  closed  form  questionnaire  is  difficult  to  prepare.  The 
designer  must  be  sure  to  include  all  possible  responses  that  he  expects  from  any  of  the 
soldiers  who  will  complete  the  questionnaire.  The  items  must  be  constructed  so  that 
they  clearly  communicate  to  the  user  exactly  what  die  designer  is  trying  to  ask.  The 


13 


JOB  INVENTORY 

(DUTY  TASK  USTI 


1.  Chock  task*  you  parform  now 

3.  Add  any  \t\k\  you  do  now  arfiich  lit  not  listed. 

3.  In  th*  "Tim*  Spant"  column,  r*t*  clocked  (^^Maski  on  tim*  spent  »n  your  pratant  job. 


P*9«  7  ol  M  P*9*i 


Tim*  Spent  Suit 

t  -  VERY  MUCH  SE LOW  AVERAGE 

4  -  AIOOT  AVERAGE 

7  -  VERY  MUCH  ABOVE 

.  2  -  91  LOW  AVERAGE 

9  -  SLIGHTLY  AIOVE  AVERAGE 

AVERAGE 

3  -  slightly  below  average 

A  -  AtOVt  AVtHAGt 

- ", — «-< — . - 

C.  FITTING  ANO  MAINTAINING  LIFE  HAFTS  AND  PRESERVERS 


t.  Ckw  itf*  pNomn 


3.  Ctaan  111*  rafts 


X  Condemn  non-*  parabta  til*  r*!t*  or  lUa  prtirwt 


4.  Fit  IHtRWrwn 


•.  IftOMCt  !»*•  prasarvars 


9.  In  poet  III*  r.dt  McmofiM 


7.  Inppact  lit*  r*lit 


S.  Impact  or  wsifht  t*«t  cartoon  dioxida  (CO  j)  cvlindars  ©t  camidytt 


t,  Mrka  antrlat  on  or  ravlasv  Uf*  Fresaryar  Data  forms  (AFTO  Form  468) 


10.  Maka  *ntrl*t  on  o«  ravlaw  Lit*  Pra*srv*  Impaction  Out*  Racord  lorn*  (AFTO  Form  3361 


It.  Mat#  antrlat  on  or  raviawr  Lila  Halt  Impaction  Racord  forma  (AFTO  Form  337) 


t3.  M»ki  antriat  on  or  rave*  L»tar  Ctnlficatinn  Lahai  forms  (AFTO  Form  27) 


t3.  Pack  lift  prasarvars 


t4.  Pock  III*  raft  aocaf  tory  containers 


IF.  Pack  HI*  rafts 


t6.  Parform  functional  tarn  ol  HI*  prasarvars 


17.  Parform  functional  tasts  of  lifa  rafts 


t6.  Parform  Inflation  tasts  of  Ufa  prasarvars 


19.  Parform  Inflation  tasts  Of  lifa  rafts 


20.  Parform  minor  repairs  to  lit*  prasarvars  such  as  patdtmy  rips,  taars,  or  holes 


CHECK  IF  I  TIME  SPENT 
DONE  IN  I  DOING  THESE 
PRESENT  1  TASKS  IN 
PRESENT  JOB 


(Continued  mat  pa*) 


greatest  single  problem  with  research  methods  is  improperly  worded  questionnaires,  as 
they  produce  faulty  data.  If  you  intend  to  design  your  own  questionnaires  we  suggest 
consulting  the  following  guides: 

Morsh,  J.E.  and  Archer,  W.B.  Procedural  guide  for  conducting  occupational 
surveys  in  the  United  States  Air  Force  (PRL-TR-67-11).  Lackland  Air  Force 
Base,  Texas:  Personnel  Research  Laboratory,  Aerospace  Medical  Division, 
Air  Force  Systems  Command,  September  1967. 

Jacobs,  T.O.  Developing  questionnaire  items :  how  to  do  it  well.  Human 
Resources  Research  Organization  (HumRRO),  300  North  Washington  Street, 
Alexandria,  Virginia  22314. 


WHAT  SHOULD  BE  INCLUDED  ON  THE  QUESTIONNAIRE? 

The  details  of  the  forms  you  use  will  be  determined  by: 

1.  how  you  will  tabulate  and  summarize  the  results,  and 

2.  what  information  you  wish  to  collect. 

How  you  will  tabulate  and  summarize  results  will  be  determined  by  whether  you 
have  available  a  computer  and  other  automated  data  handling  equipment  and  by  the 
number  of  people  surveyed.  To  determine  what  information  you  wish  to  collect,  you 
should  consider  the  total  data  requirement  for  the  training  development  process.  These 
data  requirements  should  be  piedetermined  in  the  job  analysis  plan  so  that  as  much 
information  as  possible  can  be  obtained  in  a  single  questionnaire  survey  effort. 

One  note  of  caution  about  the  design  of  vour  questionnaire  is  that  you  should  keep 
the  questionnaire  as  short  as  practical.  In  general,  the  forms  should  be  designed  so  they 
can  be  completed  in  two  hours  or  less.  One  way  you  can  save  time  on  a  complex  task 
inventory  is  to  list  all  tasks  under  their  appropriate  duty  position  title.  This  will  permit 
the  soldier  to  rspidly  scan  groups  of  tasks  that  she/he  does  not  perform  and  then  proceed 
to  the  next  duty  position. 


WHAT  TYPE  OF  INSTRUCTIONS  IFOR  COMPLETING  AND 
ADMINISTERING  THE  QUESTIONNAIRES  SHOULD  BE  PREPARED? 

After  the  formal  field  survey  questionnaires  have  been  written,  the  instructions  for 
completing  and  administering  the  questionnaires  should  be  prepared.  These  instructions 
should  include: 

•  For  the  user  —  an  introduction  explaining  the  purpose  and  impor¬ 

tance  of  participating  in  the  field  survey. 

—  general  instructions  explaining  how  the  questionnaire 
is  to  be  completed. 

•  For  the  project  officer  —  general  instructions  regarding  his/her  responsibilities. 

(Wien  questionnaire  is  —  specific  instructions  for  administering  the  question- 
not  self-administered)  tionnaire  in  a  controlled  environment. 

For  examples  of  these  types  of  instructions,  see  pages  17-23. 


TO:  USER  OF  QUESTIONNAIRE 


INTRODUCTION 

TO  THE  NONCOMMISSIONED  OFFICER: 

This  questionnaire  is  part  of  a  field  survey  designed  to  identify  tasks  for  military 
police  training.  Its  specific  purpose  is  to  obtain  from  you,  the  Noncommissioned  Officer, 
information  on  task  criticality  and  frequency  of  performance.  Feedback  gained  from 
this  questionnaire  will  piay  a  major  part  in  redesigning  the  Noncommissioned  Officer 
Advanced  Education  System.  The  ultimate  goal  is  to  design  train.ng  so  that  it  reflects 
what  we  rave  learned  from  you  in  the  field.  This  goal  is  possible  only  with  your  full 
cooperation.  Consider  each  task  listed  in  this  questionnaire  carefully  and  give  your  best 
response.  Your  contribution  is  essential  to  a  successful  survey. 


TO:  USER  OF  QUESTIONNAIRE 


GENERAL  INSTRUCTIONS 


1.  Complete  this  survey  questionnaire  within  the  time  specified  by  your  unit  project 

officer  ai  d  return  it  to  him/her  ucor,  con.pbt'on. 

2.  Because  instructions  for  completing  each  part  of  this  survey  questionnaire  are 
different,  read  all  instructions  carefully. 

3.  Part  II  requires  that  you  supply  biographical  information.  This  information  will  be 
used  to  correlate  feedback  received  from  the  field.  Print  all  answers  in  the  spaces 
provided  on  the  appropriate  survey  questionnaire  page. 

4.  In  the  upper  right  corner  of  each  page  of  Part  III,  Task  Inventory,  of  this  survey 
questionnaire  is  a  BOOKLET  NUMBER  block.  Immediately  to  the  left  of  this 
block  is  the  individual  booklet  number.  Print  the  individual  booklet  number  in  the 
BOOKLET  NUMBER  block  on  each  page  of  the  Task  Inventory  as  demonstrated  in 
the  example. 


EXAMPLE: 


(000345) 


BOOKLET  NUMBER 

r 

0 

1 

7 

m 

4 

5  6  7 

8 

9 

0 

1 

7 

3 

• 

5  6  7 

8 

9 

0 

1 

7 

3 

4 

1  6  7 

8 

9 

Part  III,  Task  Inventory,  is  divided  into  nine  (9)  separate  sections  (Sections  I-1X). 
The  content  of  these  sections  concerns  tasks  you  may  perform  in  your  present  duty 
assignment.  You  are  asked  to  rate  each  task  in  accordance  with  three  criteria  - 
frequency  of  task  performance,  immediacy  of  task  performance,  and  importance  of 
task  to  mission  success. 

Base  all  selections  on  your  experience  in  your  present  duty  assignment. 


Column  A  requires  that  you  rate  how  often  you  perform  each  task  on  a  scale 
from  one  to  four.  The  criterion  for  this  rating  is  the  frequency  of  task  per¬ 
formance.  Those  tasks  performed  most  frequently  will  normally  be  rated  four 
while  those  tasks  not  performed  ai  all  will  be  rated  one. 


b.  Column  B  requires  that  you  determine  how  soon  you  must  be  capable  of  per¬ 
forming  each  task  after  reporting  to  your  present  duty  assignment.  The  cri¬ 
terion  for  this  rating  is  the  immediacy  of  task  performance.  Of  the  four 
possible  responses,  select  the  one  most  nearly  describing  your  requirements. 
Select  response  number  four  for  those  tasks  which  you  must  be  capable  of 
performing  immediately  upon  reporting  for  duty.  Select  response  number  cme 
for  those  tasks  which  you  never  perform. 

c.  Column  C  requires  that  you  describe,  in  your  opinion,  how  important  each 
task  is  to  mission  success.  The  criterion  for  this  rating  is  the  importance  of 
the  task  to  the  accomplishment  of  the  unit  mission.  Those  tasks  that,  in 
your  opinion,  are  most  important  to  mission  success  will  be  rated  four  while 
those  tasks  that  you  consider  least  important  will  be  rated  one. 

6.  After  selecting,  enter  your  responses  for  each  task,  using  either  a  pen  or  pencil,  in 
the  answer  portion  adjacent  to  the  appropriate  task  statement  as  demonstrated  in 
the  example. 

EXAMPLE:  The  task  PREPARE  CORRESPONDENCE,  if  rated  as  performed 

FREQUENTLY  in  Column  A,  identified  as  must  be  capable  of  performing 
IMMEDIATELY  in  Column  B,  and  determined  by  you  to  be  MOST 
IMPORTANT  in  Column  C,  would  be  entered  in  the  answer  portion,  as 
shown  below. 


PREPARE  CORRESPONDENCE  123412341234 


□□□■□□□■□□01 


After  each  section  of  Part  III,  Task  Inventory,  is  a  Write-In  Section.  These  write-in 
sections  are  provided  in  order  that  you  may  comment  on  each  task  inventory  section, 
or  list  any  task(s)  you  think  should  be  included  in  the  Task  Inventory.  These  sections 
also  allow  you  to  comment  on  those  tasks  that  you  find  are  the  most  difficult  for 
you  to  perform. 
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TO:  PROJECT  OFFICER 


GENERAL  INSTRUCTIONS  FOR  PROJECT  OFFICER 


1.  General.  The  Military  Police  School  is  currently  involved  in  redesigning  basic 
military  police  training  to  produce  military  policemen  better  equipped  to  perform 
when  they  reach  the  unit.  The  emphasis  is  toward  training  replacements  in  tasks 
actually  being  performed  in  the  field.  The  questionnaires  in  this  packet  are  designed 
to  identify  those  tasks. 

The  care  with  which  you,  the  project  officer,  administer  the  questionnaires  will 
determine  the  accuracy  of  field  feedback  and,  consequently,  the  success  or  failure 
of  this  project. 

2.  Survey  Packet  Contents. 

a.  Questionnaire 

b.  Supervisor  Questionnaire 

c.  Project  Officer  Instructions 

d.  Answer  Sheets  for  Questionnaire 

e.  Pencils  for  use  on  answer  sheets. 

3.  Responsibilities. 

a.  Unit  Commander.  The  unit  commander  is  requested  to  appoint  a  project  officer 
and  to  monitor  administration  of  the  survey. 

b.  Project  Officer.  The  project  officer  is  responsible  for  the  control  and  handling 
of  questionnaires,  for  the  administration  of  the  questionnaires,  and  for  returning 
completed  and  unused  questionnaires  to  the  Military  Police  School. 

c.  Questionnaire  Administrator.  The  project  officer  may  appoint  someone  to 
administer  the  questionnaire,  if  necessary.  Normally,  however,  it  is  recom¬ 
mended  that  the  project  officer  administer  the  questionnaire. 

4.  About  the  Questionnaires. 

a.  Questionnaire.  This  questionnaire  is  designed  to  identify  tasks  being  performed 
by  military  policemen  in  the  field  and  the  frequency  with  which  each  task 

is  performed. 

b.  Supervisor  Questionnaire.  The  supervisor  questionnaire  is  programmed  to  pro¬ 
vide  feedback  on  task  criticality,  probability  of  deficient  performance,  and  the 
frequency  with  which  each  task  is  performed. 


5.  Who  Takes  The  Questionnaire.  The  project  offic  er  is  responsible  for  selecting 
individuals  to  take  the  questionnaires  (respondents)  within  their  units.  Those 
selected  must  meet  the  requirements  listed  below: 

a.  The  questionnaire  respondent  must: 

(1)  Be  in  an  M.P.  duty  assignment  (actually  performing  M.P.  duties) 

(2)  Have  been  on  the  job  at  least  90  days 

b.  The  respondent  to  the  Supervisor  Questionnaire  must: 

(1)  Command  or  supervise  M.P.  personnel 

(2)  Have  been  in  a  command  or  supervisory  position  in  the  unit  for  90  days 
(Assign  questionnaires  proportionately  among  officers  and  NCO’s) 

6.  Questionnaire  Administration. 

a.  Questionnaire.  The  questionnaire  will  be  administered  in  a  controlled  environ¬ 
ment.  Persons  participating  in  the  survey  will  be  allowed  two  hours  to  complete 
the  questionnaire  and  will  turn  the  questionnaire  and  answer  sheets  in  to  the 
questionnaire  administrator  prior  to  leaving  the  survey  area.  Individual  question¬ 
naires  and  their  accompanying  answer  sheets  will  be  kept  together. 

See  attached  item  for  the  procedure  to  be  followed  in  administering  the 
questionnaire. 

b.  Supervisor  Questionnaire.  Supervisors  selected  as  respondents  for  this  ques¬ 
tionnaire  will  be  allowed  to  sign  for  the  questionnaire  and  take  it  with  them. 
They  will  complete  the  questionnaire  and  return  it  to  the  project  officer  within 
a  time  frame  he/she  specifies.  This  time  frame  must  be  compatible  with  the 
suspense  date  to  the  Military  Police  School. 

7.  Questionnaire  Handling.  Questionnaires  and  answer  sheets  become  FOR  OFFICIAL 
USE  ONLY  when  completed.  For  ease  of  accounting,  each  questionnaire  and  its 
accompanying  answer  sheets  are  numbered.  All  questionnaires  must  be  returned  to 
the  Military  Police  School  whether  they  are  used  or  not.  Instructions  for  returning 
the  questionnaires  to  the  Military  Police  School  are  contained  in  the  basic  letter. 

If  you  have  any  problems  or  questions,  contact  (NOTE:  Give  name  or  names, 
address,  and  telephone  number). 
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TO:  PROJECT  OFFICER  ADMINISTERING  QUESTIONNAIRE  IN  A  CONTROLLED 
ENVIRONMENT 


ADMINISTERING  THE  QUESTIONNAIRE 


Preparation,  A  classroom  or  training  room  equipped  with  desks  will  provide  the 
most  ideal  site  for  administering  the  questionnaire.  Questionnaires,  answer  sheets, 
and  two  electrographic  pencils  should  be  issued  to  participants  after  everyone  who 
is  to  take  the  questionnaire  has  arrived.  This  ensures  that  everyone  starts  together. 

Instructions.  The  questionnaire  administrator  will  present  the  following  instructions. 

a.  “Is  there  anyone  here  who  is  not  working  in  an  M.P.  duty  position?  Is  there 
anyone  here  who  has  not  been  assigned  to  their  present  duties  at  least  90  days? 
If  so,  please  leave  at  this  time.” 

b.  “Will  everyone  at  this  time  please  read  the  first  page  in  the  questionnaire  which 
has  been  issued  to  you.” 

(Note  to  the  administrator:  It  must  be  emphasized  that  your  enthusiasm  for 
this  project  or  lack  of  it  will  be  contagious.  It  is  important  that  you  demon¬ 
strate  a  positive  attitude  to  the  participants.  Allow  time  for  the  first  page  to 
he  read  and  underline  the  importance  of  the  questionnaire  with  the  following 
statement.) 

“I  would  like  to  stress  the  importance  of  this  questionnaire.  The  Military 
Police  School  wants  to  design  training  to  fit  the  job  in  the  field.  You  are 
the  only  people  who  can  tell  them  what  they  need  to  know.  Please  think 
through  each  question  and  give  your  best  answers.” 

c.  “Turn  to  Part  1,  Biographical  Information,  and  answer  questions  1-13. 

When  you  have  finished,  lay  your  pencil  on  the  desk  so  I  will  know  when 
to  proceed  to  the  next  step.” 

d.  “Now  read  the  instructions  found  in  Part  II.” 

(Note  to  the  administrator:  Allow  reasonable  time  for  everyone  to  finish  before 
moving  to  the  next  step.) 

“Are  there  any  questions?” 

e.  “As  you  read  in  the  instructions,  there  are  nine  answer  sheets  accompanying 
your  questionnaire.  Take  the  answer  sheets  and  number  them  one  through 
nine  to  correspond  to  the  first  nine  sections  in  Part  III  of  the  questionnaire. 

If  you  do  not  have  nine  answer  sheets,  raise  your  hand— I  have  extra  ones. 

Use  a  separate  answer  sheet  for  each  section.  Answer  only  the  number  of 
questions  listed  in  each  and  move  to  the  next  section  and  answer  sheet.  It 
is  not  necessary  to  write  your  name,  rank,  the  date,  or  course  at  the  top 

of  the  answer  sheet.  Also,  disregard  the  blocks  marked  score,  grade,  extra 
points,  and  social  security  number.” 
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f.  “Because  of  the  size  of  this  survey,  these  answer  sheets  will  be  read  by 
machine.  You  must  vse  the  special  pencils  provided  so  that  the  machine  can 
read  the  answers.  When  marking  your  answer,  take  care  to  fill  the  vertical 
rectangle  outlining  the  letter  as  shown  by  the  example  on  page  4  of  the 
instructions.  Also,  piease  be  sure  not  to  make  any  stray  marks  on  the 
answer  sheets.  Finally,  do  not  fold  the  answer  sheets.” 

g.  “All  answers  must  be  based  on  your  experience  in  this  your  present  unit.  Do 
not  call  on  experience  in  previous  units.  This  means  that  if  you  do  not  per¬ 
form  a  particular  task  in  your  present  unit,  you  must  mark  ‘do  not  perform 
this  task’ on  your  answer  sheet.” 

h.  “You  may  begin  answering  Part  III.  Remember  Section  ten,  the  written 
section.  When  you  finish  answering  all  questions,  insert  your  answer 
sheets  into  the  questionnaire  and  turn  them  in  to  me.  You  may  leave 
when  you  are  finished.  Are  there  any  questions?” 

A-3.  Conclusion.  After  everyone  has  taken  the  questionnaire,  ensure  that  all  ques¬ 
tionnaires  and  answer  sheets  are  accounted  for.  Collect  the  pencils  provided  so 
that  they  may  be  returned  to  the  Military  Police  School  along  with  the  question¬ 
naires  and  answer  sheets. 
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HOW  IS  A  SURVEY  SAMPLE  SELECTED? 


You  are  now  ready  to  select  organizations  and  individuals  to  provide  you  with  the 
needed  job  data.  The  complexity  of  the  MOS,  the  availability  and  quality  of  publish'd 
sources  of  job  information,  and  the  number  of  people  in  the  particular  job  will  determine 
how  much  and  what  kinds  of  information  you  need  to  collect.  This  will  strongly 
influence  the  make-up  and  size  of  your  sample.  If  the  complexity  and  amount  of  required 
data  are  great,  the  number  of  organizations  and  individuals  interviewed  will  increase.  As 
a  general  rule,  your  survey  sample  should  be  as  large  as  possible.  This  is  particularly  true 
if  you  do  not  have  personnel  available  with  the  responsibility  and  authority  to  assure  that 
most  of  your  questionnaires  will  be  completed.  You  should  make  an  attempt  to  obtain  a 
sample  that  represents  the  distribution  of  individuals  in  the  MOS  ar  >rding  to  command 
and  skill  level.  Review  of  personnel  records,  either  by  personnel  e.n-Moyees,  members  of 
your  job  analysis  team,  or  your  field  represen  tatives  who  will  conduct  the  survey  will  be 
required  to  obtain  data  upon  which  to  base  choices  for  your  survey. 

•  In  selecting  UNITS  for  sampling,  you  should  select  units  that: 

1.  have  at  least  a  small  number  of  job  holders  and  supervisors  who  do  the 
particular  job  to  be  analyzed.  Preferably,  you  should  choose  some  units 
that  have  a  relatively  small  number  of  job  holders,  and  some  that  have 
large  numbers. 

2.  are  geographically  and  environmentally  representative. 

«  In  selecting  INDIVIDUALS  within  the  units,  you  should  select  a  group  made  up 
of  individuals  who: 

1.  perform  and  supervise  the  job  being  analyzed. 

2.  perform  with  average  satisfactory  proficiency. 

3.  are  representative  in  terms  of  length  of  time  on  the  job. 

4.  are  representative  in  terms  of  training. 

For  certain  types  of  information  you  will  also  want  to  choose  at  least  a  few  job  holders  or 
supervisors  who  are  acknowledged  experts  at  the  job. 

HOW  IS  A  QUESTIONNAIRE  SURVEY  CONDUCTED? 

•  Trial  Run  (Validation  of  Instruments) 

Before  sending  out  the  total  number  of  questionnaires  you  intend  using,  you  may 
wish  to  send  out  a  small  number.  This  will  permit  you  to  check  the  initial  results  and 
possibly  make  some  changes  in  your  questionnaires  or  instructions.  Then  you  will  send 
out  what  you  hope  will  be  the  total  number  cf  questionnaires  required. 


•  Group  Administration 

The  ideal  way  to  administer  questionnaires  is  group  administration.  Where  the 
local  responsible  official  and  his/her  assistants  schedule  the  administration  he/she  should 
do  the  following: 

—  Make  certain  that  only  eligible  individuals  are  seated  in  the  administration  area. 
—  Read  the  administrative  instructions. 

—  Provide  any  necessary  assistance  in  completing  the  questionnaires. 

—  Return  the  completed  questionnaires  to  the  school. 


•  Individual  Administration 

Often,  particularly  with  individuals  at  remote  stations,  group  administration  is 
impractical.  In  these  cases,  it  is  sometimes  effective  to  send  the  questionnaires  to  a 
responsible  officer  and  request  that  he/she  return  them  by  a  reasonable  suspense  date. 
Careful  attention  should  be  paid  to  the  instructions  for  administration  of  self-administration. 
If  your  command  has  no  authority  to  require  that  a  suspense  date  be  met,  then  you  must 
either  obtain  the  concurrence  of  a  command  with  that  authority,  or  be  willing  to  accept 
a  reduced  percentage  return. 


•  Return  of  Questionnaires 

How  much  confidence  can  you  nave  in  the  validity  of  your  questionnaire  if  you 
get  less  than  a  100  percent  return?  Less  and  less  confidence  can  be  expected  with  each 
reduction  in  the  percentage  returned.  What  can  you  do  if  you  are  not  satisfied  with  the 
percentage  of  returns  of  the  questionnaires?  We  suggest  you  try  the  following: 

1)  Send  out  more  forms  to  different  people  and  hope  for  better  results. 

2)  Recontact  some  of  the  first  sample  and  try  to  encourage  them  to  return 
the  questionnaires. 

3)  Visit  a  random  sample  of  those  who  did  not  respond  and  compare  their 
forced  responses  with  the  voluntary  responses.  Then  you  and  your  super¬ 
visors  will  have  to  decide  how  much  chance  you  are  willing  to  take  (hat 
the  data  you  have  received  presents  a  sufficiently  accurate  picture  of  the 
job  as  >t  really  exists. 


Chapter  IV 

ORGANIZING  PANELS  OF  EXPERTS 


WHAT  IS  A  PANEL  OF  EXPERTS? 

With  this  method  a  group  of  personnel,  selected  for  their  experience  and  knowledge 
of  the  job,  is  brought  together  to  confer  about  the  required  job  analysis  data.  Panels 
may  be  made  up  of  one  or  more  of  the  following  types  of  members: 

1)  Subject  Matter  Experts  (SMEs) 

These  are  personnel  found  at  your  school  who  are  acknowledged  experts 
in  the  tasks,  duty  positions  or  MOS  you  are  analyzing.  They  may  be 
found  among  instructors  or  ISD  personnel  who  hold  the  MOS.  SMEs 
may  or  may  not  have  had  recent  field  experience. 

2)  Job  Incumbents 

This  group  includes  those  who  are  presently  holding  the  jobs/performing 
the  tasks  you  are  interested  in,  or  who  have  recently  held  the  jobs.  The 
more  recent  the  better.  More  than  three  years  away  from  the  field  would 
disqualify  a  potential  panel  member. 

3)  Job  Supervisors 

This  group  includes  those  who  are  presently  or  have  recently  (within  the 
past  three  years)  supervised  soldiers  in  the  jobs/tasks  you  are  interested  in. 

Job  incumbents  and  job  supervisors  may  be  found  on  the  post  where  your  school  is 
located  or  at  other  locations.  Check  the  TOEs  and  TOAs  to  find  out  where  personnel 
may  be  assigned.  While  personnel  assigned  to  your  post  are  most  conveniently  assembled, 
they  may  not  be  completely  representative  of  job  incumbents/supervisors  serving  in  other 
locations.  Also,  they  probably  have  been  heavily  burdened  by  school  surveys,  panels,  etc,, 
already  due  to  their  proximity  to  the  school,  Therefore,  personnel  from  other  locations 
may  have  to  be  used. 


WHEN  IS  A  PANEL  USED? 

Panels  of  experts  are  a  good  alternate  data  source  when: 

—  CODAP  is  not  available. 

—  time  and  funds  art-  inadequate  for  conducting  a  field  survey. 

—  the  type  of  information  required  can  be  reliably  provided  by  a  small  group. 
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WHEN  ARE  SUBJECT  MATTER  EXPERTS  USED? 


With  this  method,  a  group  of  highly  experienced  personnel  is  brought  together  to 
record  and  organize  the  required  job  analysis  data.  This  method  is  particularly  useful  in 
collecting  job  data  on  new  jobs  or  on  managerial  and  supervisory  jobs  where  many  of 
the  most  critical  behaviors  are  not  directly  observable.  Since  the  members  of  a  panel  of 
SMEs  are  experts  in  the  MOS  being  analyzed,  their  collective  effort  should  be  decisions 
about  the  requirements  of  the  job.  In  general,  their  greatest  effectiveness  is  in  evaluating 
and  making  decisions  about  job  data  that  have  been  collected  from  other  sources  by 
other  means. 


WHEN  ARE  RECENT  JOB  INCUMBENTS  AND/OR  SUPERVISORS  USED? 

With  this  method,  a  group  of  job  incumbents,  job  supervisors,  or  a  combination  of 
these,  is  brought  together  to  provide  information  about  their  jobs.  The  primary  function 
of  this  type  of  panel  is  to  provide  information  about  their  jobs,  not  to  make  decisions. 
Another  name  for  this  type  of  panel  is  Consensus  Group. 


HOW  IS  A  PANEL  ASSEMBLED? 

The  panel  is  a  relatively  inexpensive  and  easy  approach  to  collecting  data.  Three  to 
seven  persons  is  the  number  recommended  to  make  up  the  panel.  Whenever  possible,  you 
want  panel  members  to  be  representative  of  different  locations  and  types  of  units  in  the 
field.  While  many  experts  may  be  available  within  your  school  it  is  critical  that  their 
views  be  balanced  by  those  of  persons  serving  presently  or  recently  in  the  field.  If  ail  of 
the  panel  members  come  from  schools,  there  may  he  a  tendency  for  the  outcome  to  reflect 
what  exists  in  training  rather  than  what  actually  exists  on  the  job. 
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